Technology exists for illuminating a user's eye with infrared or other non-visible light and detecting the gaze direction by analyzing images of light reflected from the user's eye in Japanese Patent Application No. H02-264632. The eye-gaze detection technique is known as analyzing the captured emitted light images of user's eye using illuminating light on the user's eye. The detected eye-gaze information can be used, for example, for PC (Personal Computer) or game monitoring, and can be applied to use as a pointing device.
The head mounted display is wearable on the user to display 3D images but normally covers the user's eye sights. The user therefore cannot recognize the outside environment while the user wears the HMD. If the user needs to find input devices such as a controller while wearing the HMD as a display device for movie or games, the user has difficulties to find controllers.
It could therefore be helpful to detect the user's eye-gaze direction to use as a substitute of pointing device while the user wears the HMD. Thus, it could be helpful to provide a technique that detects the gaze direction of a user wearing a head mounted display.
We provide a gaze detection system comprising: a head mounted display, mounted on a user's head for use, comprising, a first infrared light source unit illuminating the user's right eye with infrared light; a second infrared light source unit illuminating the user's left eye; a first image capturing device imaging the right eye illuminated by the infrared light source; a second image capturing device imaging the left eye illuminated by the infrared light source; and a display unit displaying a three-dimensional image, and a gaze detection device, detecting the gaze of the user, comprising, a first detection unit detecting the gaze direction of the right eye, based on the image captured by the first image capturing device; a second detection unit detecting the gaze direction of the left eye, based on the image captured by the second image capturing device; and a tracking unit determining the gaze point of the user in the three-dimensional image on the basis of the gaze direction of the right eye and the gaze direction of the left eye.
Also, the first detection unit may calculate the right eye gaze vector indicating the gaze direction of the right eye, and the second detection unit may calculate the left eye gaze vector indicating the gaze direction of the left eye.
The display unit may display a three-dimensional image consisting of a plurality of layers in the depth direction, and the tracking unit may identify the layer that the user is gazing at by selecting the layer having the shortest distance between the intersection points of the right eye gaze vector and the left eye gaze vector with each layer.
The tracking unit may identify the user's gaze location based on the intersection point between the right eye gaze vector and the left eye gaze vector.
The tracking unit may identify the user's gaze location based on the intersection region of a cylinder with a given radius, centered on the right eye gaze vector and a cylinder with a given radius, centered on the right eye gaze vector.
The tracking unit may identify the gaze location of the user based on the intersection points of the first plurality of parallel vectors parallel to the right eye gaze vector and the second plurality of parallel vectors parallel to the left eye gaze vector.
Additionally, any combination of the aforementioned components, and the implementation in the form of methods, devices, systems, computer programs, data structures, recording mediums, and the like may be effective.
We thus disclose a technique to detect the gaze direction of a user wearing a head mounted display.